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Description 

APPARATUS CAPABLE OF PERFORMING 

BOTH BLOCK-MATCHING MOTION 
COMPENSATION AND GLOBAL MOTION 
COMPENSATION AND METHOD THEREOF 

Background of Invention 
[000 1 ] 1. Field of the Invention 

[0002] The invention relates to digital video, and more particu- 
larly, to decoding a coded video bit stream having both 
macroblocks encoded using block-matching motion com- 
pensation and macroblocks encoded using global motion 
compensation. 

[0003] 2. Description of the Prior Art 

[0004] Full-motion video displays using analog video signals 

have long been available in the form of television. With re- 
cent advances in computer processing capabilities and af- 
fordability, full-motion video displays using digital video 
signals are becoming more widely available. Digital video 



systems provide significant improvements over conven- 
tional analog video systems in creating, modifying, trans- 
mitting, storing, and playing full-motion video sequences. 

[0005] However, the amounts of raw digital information included 
in video sequences are massive. Storage and transmission 
of these amounts of video information is infeasible with 
conventional personal computer equipment. Consider, for 
example, a digitized form of a relatively low resolution 
VHS image format having a 320x480 pixel resolution. A 
full-length motion picture of two hours in duration at this 
resolution corresponds to 100 gigabytes of digital video 
information. By comparison, conventional CD-ROM disks 
have capacities of about 0.7 gigabytes, and DVD disks 
have capacities of up to 8 gigabytes. 

[0006] jo address the limitations in storing and transmitting 

such massive amounts of digital video information, vari- 
ous video compression standards or processes have been 
established, including MPEG-1, MPEC-2, MPEG-4, and 
H.26X. These video compression techniques utilize still 
image compression techniques, referred to as intraframe 
correlation, of the individual image frames as well as sim- 
ilarities between successive image frames, referred to as 
interframe correlation, to encode the digital video infor- 



mation and provide a high compression ratio. 
[0007] Block-matching (BM) motion compensation is a technique 
well known in the prior art for encoding digital video in- 
formation. If an image sequence shows moving objects, 
then their motion within the sequence can be used to cre- 
ate a motion vector for a particular block containing the 
moving object, also referred to as a macroblock. This mo- 
tion vector can be used to predict where the macroblock 
will be later in the sequence. Instead of transmitting a new 
image, the motion vectors for macroblocks containing the 
moving objects can be sent instead. Block-matching mo- 
tion compensation greatly reduces the data that must be 
transmitted for image sequences containing moving ob- 
jects. However, when the whole image is panning, ex- 
panding, contracting, or turning, the motion vectors of all 
of macroblocks must be transmitted, greatly decreasing 
the coding efficiency. To solve this problem, global mo- 
tion compensation (GMC) techniques are well known in 
the prior art, such as the "sprite" coding techniques used 
in MPEG-4 (i.e. ISO/IEC 14496-2). These global motion 
compensation techniques take into account global image 
changes between a previous frame and the current frame. 
Global motion parameters associated with each frame are 



used to specify individual motion vectors for all pixels in 
each macroblocks encoded using global motion compen- 
sation. In this way, only one set of global motion parame- 
ters is required for each frame, increasing the encoding 
efficiency for video sequences having global image 
changes. 

[0008] pig.l shows a typical video decoder 100 according to the 
prior art as disclosed in US Patent No. 6,483,877. The 
video decoder 100 receives a incoming coded video bit 
stream 102 that is separated through a demultiplexerl04 
into quantized discrete cosine transform (DCT) coeffi- 
cientsl06, macroblock motion vector and global motion 
parameters 108, and an intraframe/interframe distinction 
flag 110. The quantized DCT coefficient 106 is decoded 
into an error image 116 through an inverse quantizer 112 
and an inverse DCT processor 114. An output image 118 
of an interframe/intraframe switching unit 120 is added to 
the error image 116 through an adder 122 to form a re- 
constructed image 124. 

[0009] The interframe/intraframe switching unit 120 switches its 
output 118 according to the interframe/intraframe coding 
distinction flag 110. A predicted image synthesizer 126 
synthesizes a predicted image 128 that is used for exe- 



cuting the interframe coding. The predicted image syn- 
thesizer 126 performs motion compensation operation 
and fetches prediction blocks from at least one decoded 
image 130, which is a previously decoded frame stored in 
a frame memory 128. The predicted image synthesizer 
126 performs either block-matching motion compensa- 
tion or global motion compensation according to the en- 
coding type used for a particular macroblock. In the case 
of intraframe coding, the interframe/intraframe switching 
unit 120 outputs the "0" signal 132 and the output of the 
predicted image synthesizer 126 is not used. 
[0010] pig. 2 shows a more detailed block diagram of the pre- 
dicted image synthesizer 126 of Fig.l according to the 
prior art. The predicted image synthesizer 126 processes 
global motion compensation and block matching motion 
compensation in parallel. The macroblock motion vector 
and global motion parameters 108are input to ademulti- 
plexer 202, which provides global motion parameters204, 
a macroblock motion vector 206, and a selection signal 
208 specifying block matching/global motion compensa- 
tion to a GMC image synthesizer 210, a BM image synthe- 
sizer 212, and a switch 214, respectively. The BM image 
synthesizer 212 synthesizes the predicted image for 



blocks that are encoded using block-matching motion 
compensation, and the GMC image synthesizer 210 syn- 
thesizes the predicted image for blocks that are encoded 
using global motion compensation. The respective pre- 
dicted image data 216 and 218are output to the switch 
214, which selects one of these signals according to the 
selection signal208, received from the demultiplexer 202. 
The predicted image 128 is then output to the switching 
unit 120, as shown in Fig.l. 
1 ] As can be seen from the above description, video de- 
coders supporting both block-matching motion compen- 
sation and global motion compensation require the use of 
two different image synthesizers. A first image synthe- 
sizer 212 is used for block-matching motion compensa- 
tion, and a second image synthesizer 210 is used for 
global motion compensation. When processing blocks en- 
coded using block-matching image compensation, the 
GMC image synthesizer 210 is idle. Likewise, when pro- 
cessing blocks encoded using global motion compensa- 
tion, the BM image synthesizer 212 is idle. This non- 
optimal solution of requiring two image synthesizers in- 
creases the hardware complexity of the video decoder and 
results in a higher cost. It would be beneficial to combine 



the functionality of the GMC synthesizer 210 and the BM 

synthesizer 212 into an integrated unit. 
Summary of Invention 

[0012] it j S therefore an objective of the invention to provide an 
apparatus capable of performing motion compensation 
for both macroblocks encoded with block-matching and 
macroblocks encoded with global motion compensation, 
to solve the above-mentioned problems. 

[0013] According to one embodiment of the invention, an appa- 
ratus is disclosed for performing motion compensation. 
The apparatus is capable of decoding an incoming coded 
video bit stream including a plurality of frames. Each 
frame may include macroblocks encoded using block- 
matching motion compensation and/or macroblocks en- 
coded using global motion compensation. The apparatus 
further includes an interpolation unit. The interpolation 
unit performs interpolation operations on each mac- 
roblock encoded with block-matching or global motion 
compensation in each frame of the incoming coded video 
bit stream. When processing a current macroblock, if the 
current macroblock is encoded using global motion com- 
pensation, the interpolation unit performs the interpola- 
tion operations according to a global motion vector trans- 



lated from the global motion parameters. 

[0014] Also according to the present invention, a predicted image 
synthesizer in a video decoder is disclosed for decoding a 
video bit stream and generating a predicted image. The 
video bit stream includes a plurality of frames having first 
macroblocks encoded using block-matching compensa- 
tion and second macroblocks encoded using global mo- 
tion compensation. The video bit stream includes mac- 
roblock motion vectors indicating motion vectors of the 
first macroblocks and global motion parameters associ- 
ated with the plurality of frames indicating a motion vec- 
tor of each pixel in the second macroblocks. The pre- 
dicted image synthesizer comprises a translation unit re- 
ceiving the global motion parameters, and translating the 
global motion parameters into a global motion vector 
which is in a form substantially identical to that of the 
macroblock motion vector, and a interpolation unit for re- 
ceiving at least one prediction block in a decoded image 
130, which is a previously decoded frame, receiving the 
global motion vector, performing interpolation operations, 
and generating the prediction image. 

[0015] These and other objectives of the claimed invention will 
become obvious to those of ordinary skill in the art after 



reading the following detailed description of the preferred 

embodiment that is illustrated in the various figures and 

drawings. 
Brief Description of Drawings 

[0016] pig.l is a typical video decoder according to the prior art. 

[0017] Fig.2is a block diagram of the predicted image synthesizer 
of Fig.l according to the prior art. 

[0018] pig. 3 is a block diagram of a predicted image synthesizer 
according to the present invention. 

[0019] Fig. 4 is a diagram showing half-pel (half pixel) interpola- 
tion for luminance and chrominance block-matching 
compensation performed by the predicted image synthe- 
sizer of Fig. 3. 

[0020] Fig. 5 is a matrix showing half-pel (half pixel) interpolation 
for the diagram of Fig. 4. 

[0021] Fig. 6 is a diagram showing global motion compensation 
according to the prior art. 

[0022] Fig. 7 is a matrix showing half-pel (half pixel) interpolation 
for luminance global motion compensation performed by 
the predicted image synthesizer of Fig. 3. 

[0023] Fig. 8 is a matrix showing quarter-pel (quarter pixel) inter- 
polation for chrominance global motion compensation 



performed by the predicted image synthesizer of Fig. 3. 

[0024] pig. 9 is a flowchart describing a method of processing an 

incoming coded video bit stream including a plurality of 

frames according to the present invention. 
Detailed Description 

[0025] pig. 3 is a block diagram of a predicted image synthesizer 
300 according to the present invention. The predicted im- 
age synthesizer 300 can be used in a video decoder such 
as the video decoder 100 shown in Fig.l. In Fig. 3, all sig- 
nals that contain the same information as signals in Fig.l 
are labeled using the same numerical labels as in Fig.l. 
The predicted image synthesizer 300 includes a demulti- 
plexer 302, a translation unit 304, a macroblock MV 
(motion vector) storage unit 306, a global MV (motion 
vector) storage unit 308, a switching unit 310, and an in- 
terpolation unit 312. The demultiplexer 302 receives the 
macroblock motion vector and the global motion parame- 
ters 108 derived from the incoming coded video bit 
stream. At the beginning of each frame, the global motion 
parameters 307 are passed to the translation unit 304. 
The translation unit 304 converts the global motion pa- 
rameters 307 into a global motion vector for luminance 
(luminance global motion vector) and a global motion 



vector for chrominance (chrominance global motion vec- 
tor). The luminance global motion vector and the chromi- 
nance global motion vector can be used during interpola- 
tion operations at the macroblock level for all mac- 
roblocks encoded using global motion compensation 
throughout the frame. The luminance global motion vec- 
tor and the chrominance global motion vector are stored 
in the global MV storage unit 308. For all macroblocks 
that are encoded using block-matching motion compen- 
sation, at least one macroblock motion vector is included 
within the macroblock. The macroblock motion vector 305 
is contained within the macroblock motion vectors and 
global motion parameters 108, and is passed to the pre- 
dicted image synthesizer 300. The macroblock motion 
vector 305 is then stored in the macroblock MV storage 
unit 306. According to the MPEC-4 specification, the mac- 
roblock motion vector 305 is actually a macroblock mo- 
tion vector for luminance (luminance macroblock motion 
vector). A macroblock motion vector for chrominance 
(chrominance macroblock motion vector) can be obtained 
by performing calculation on the luminance macroblock 
motion vector. To save the space of the Macroblock MV 
storage 306, only the macroblock motion vector 305 (that 



is, the luminance macroblock motion vector) is stored in 
the Macroblock MV storage 306. The chrominance mac- 
roblock motion vector will be calculated and obtained in 
the Interpolation unit 312 in this embodiment. It should 
be noted that the macroblock MV storage unit 306 may 
also be replaced by a straight connection to the switching 
unit 310 in another embodiment of the present invention. 
The switching unit 310 is used to either pass the mac- 
roblock motion vector from the Macroblock MV storage 
306 or the luminance/chrominance global motion vector 
from the global MV storage unit 308 to the interpolation 
unit 312 according to the motion compensation type 316. 
[0026] The interpolation unit 312 reads at least one prediction 
block in a decoded image 130, and the location of the 
prediction block in the decoded image is determined by 
the motion vector 314 received from the switching unit 
310. The interpolation unit 312 then performs both lumi- 
nance and chrominance interpolation operations of the 
corresponding prediction block. The received motion vec- 
tor 314 may be the macroblock motion vector retrieved 
from the macroblock MV storage unit 306 or the lumi- 
nance/chrominance global motion vector retrieved from 
the global MV storage unit 308. The interpolation unit 312 



receives the decoded image 130 and the motion vector 
314, performs interpolation operations, and outputs a 
predicted image 128 that is used for executing the inter- 
frame coding in a video decoder such as the video de- 
coder 100 shown in Fig.l. The interpolation unit 312 may 
further include a buffer 313 for temporarily storing the 
interpolation operation result corresponding to the differ- 
ent macroblock motion vectors in a macroblock so as to 
properly perform the interpolation operations of a mac- 
roblock. For example, when performing bi-directional in- 
terpolation for a block matching macroblock, the interpo- 
lation unit 312 may temporarily store the forward predic- 
tion interpolation result in the buffer 313, combine with 
the later backward prediction interpolation result, and 
then obtain the final bi-directional interpolation result. 
[0027] jo further explain the luminance and chrominance inter- 
polation operations performed by the interpolation unit 
312, please refer to Fig. 4 and Fig. 5. Fig. 4 is a diagram 
showing half-pel (half pixel) interpolation for luminance 
and chrominance block-matching compensation per- 
formed by the predicted image synthesizer 300 of Fig. 3. 
Fig. 5 is a matrix showing the resulting half-pel (half pixel) 
bilinear interpolation for the diagram of Fig. 4. In Fig. 4 



there are four integer pixel positions (A, B, C, D), labeled 
in Fig. 5 as (I a , I b , l c , and I d ), as well as five half-pixel po- 
sitions, labeled in Fig. 5 as (H^ H^, H 3 , H 4 , H 5 ). When per- 
forming motion compensation for macroblocks encoded 
using block-matching motion compensation, the present 
invention uses the same half-pel (half pixel) bilinear inter- 
polation process of the prior art. To briefly describe the 
bilinear interpolation calculations, the following formulas 
show the interpolation calculations for the pixel positions: 
I , H , H 2> and (The remaining pixel positions are cal- 
culated in the same manner, as is well known in the prior 
art.) 

[0028] | = | 
A A 

[0029] H = (I + I + 1 rounding.control) / 2 

1 A B 3 

[0030] H = (I + I + 1 rounding.control) / 2 

[0031] H=(l+l+l+l+2 rounding.control) / 4 

3 A B C D 3 

[° 032 ] where the rounding.control parameter is a value of 0 
or 1, and is derived from the incoming coded video bit 
stream. 

[0033] pig. 6 shows a diagram showing global motion compensa- 
tion according to the prior art. Global motion compensa- 
tion according to the prior art involves converting the 



global motion parameters into an individual motion vector 
for each pixel in each macroblock encoded using global 
motion compensation. In Fig. 6 there are again four inte- 
ger pixel positions (Y , Y , Y , and Y ) as well as a 

a K K 00 01 10 11 

non-integer pixel position Y. The non-integer pixel posi- 
tion Y has a vertical distance of (r /s) and a horizontal po- 

j 

sition of (r /s), where s is specified by 

i 

sprite_warping_accuracy, as defined in the MPEG-4 
(ISO/IEC 14496-2) specification. The present invention 
takes advantage of the fact that when the 
no_of_sprite_warping_point parameter in MPEG-4 is set to 
a value of 0 or a value of 1, the global motion parameters 
can be converted into a global motion vector having the 
same value for all pixels in the frame. This means that in- 
stead of doing global motion compensation on a per-pixel 
bases, as is done in the prior art, the present invention 
can also perform global motion compensation on a per- 
macroblock basis using a single motion vector for each 
macroblock. In this way, almost the same hardware as is 
used in the prior art BM image synthesizer 212 can be 
used to perform both block-matching motion compensa- 
tion and global motion compensation in the present in- 
vention. 



[0034] pig. 7 shows a diagram illustrating half-pel (half pixel) bi- 
linear interpolation for luminance global motion compen- 
sation performed by the interpolation unit 312 of Fig. 3. 
When the no_of_sprite_warping_point parameter in MPEG- 
4 is set to a value of 0 or a value of 1, the luminance in- 
terpolation operations of Fig. 6 can be reduced to the ma- 
trix shown in Fig. 7. The matrix shown in Fig. 7 is equiva- 
lent to the matrix shown in Fig. 5 and this implies the lu- 
minance and chrominance interpolation operations for 
block-matching compensation and the luminance inter- 
polation operations for global motion compensation can 
be performed using the same interpolation unit 312 at a 
half-pel resolution. 

[0035] pig. 8 is a diagram showing quarter-pel (quarter pixel) bi- 
linear interpolation for chrominance global motion com- 
pensation performed by the interpolation unit 312 of 
Fig. 3. When the no_of_sprite_warping_point parameter in 
MPEG-4 is set to a value of 0 or a value of 1, the chromi- 
nance interpolation operations of Fig. 5 can be reduced to 
the matrix shown in Fig. 7. Fig. 7 is simply a quarter-pel 
matrix which is solved using the same bilinear interpola- 
tion process as the half-pel matrix but at twice the reso- 
lution. To briefly describe the interpolation calculations at 



quarter-pel, the following formulas show the interpolation 
calculations for the pixel positions: I , Q , Q 4 > Q« Q 6 > 
H , Q , and H . (The remaining pixel positions are cal- 
culated in the same manner.) 

[0036] | = | 
A A 

[0037] q = (3 | + | + 2 rounding.control) / 4 

[0038] H 2 = (I + I + 1 rounding.control) / 2 

[0039] Q = (3 I + I + 2 rounding.control) / 4 

[0040] Q=(9l+3l+3l+l+8 rounding.control) / 16 

5 A B CD a 

[0041] Q=(3l+3l+l+l+4 rounding.control) / 8 

6 A BCD 

[0042] H9 = (I + I +1 rounding.control) / 2 

c 

[0043] qio = (3 1 +1 + 31 +1 +4 rounding.control) / 8 

A B C D 3 

[0044] Hll =( I +1 +1 +1 + 2 rounding.control) / 4 

A B C D 3 

[° 045 ] where the rounding.control parameter is a value of 0 
or 1, , and is derived from the incoming coded video bit 
stream. 

[0046] it should be noted that the translation unit 304 converts 
the global motion parameters 307 received for each frame 
into a luminance global motion vector and a chrominance 
global motion vector for the frame. The following formu- 



las describe the conversion process implemented by the 
translation unit 304 to covert the global motion parame- 
ters into the luminance/chrominance global motion vec- 
tors used during interpolation operations for macroblocks 
encoded using global motion compensation with 
no_of_sprite_warping_point equal to 0 and 
no_of_sprite_warping_point equal to 1, respectively. The 
luminance global motion vector is a half-pel precision 
motion vector and the chrominance global motion vector 
is a quarter-pel precision motion vector. Hence, these 
macroblocks encoded using global motion compensation 
can be treated as if they were block matching mac- 
roblocks with prediction mode of "frame.prediction" and 
with the following luminance/chrominance global motion 
vectors respectively for the luminance/chrominance com- 
ponents. 

[0047] For GMC macroblocks with (no_of_sprite_warping_point 

= = 0), (sprite_enable == GMC), and 

(video_object_layer_shape == rectangle): 
[0048] MV GMC - Y = (MV GMC - Y , MV GMC - Y ) = (0,0) 

X y 

[0049] MV GMC - CbCr = (MV CMC - cbCr MV GMC - cbCr ) = (0,0) 

X y ' 

[0050] For GMC macroblocks with (no_of_sprite_warping_point 



= = 1), (sprite_enable == CMC), and 
(video_object_layer_shape == rectangle"): 

mr m = (Mr^.urf**) = (i 0 j' 0 ) 

= ((s/2)dti[0],(sJ2)d^0]) 
wkere(i 0 j' Q ) = ((s f2)du[0],(sf 2)^v[0]) 

[0051] where du[0] and dv[0] are the global motion parameters 
derived from the incoming coded video bit stream and s is 
specified by sprite_warping_accuracy, as defined in the 
MPEG-4 (ISO/IEC 14496-2) specification. 

[0052] it i S noted that the above equations apply when 

(no_of_sprite_warping_point == 0 or 1), (sprite.enable = = 
GMC), and (video_object_layer_shape == rectangle). But 
the present invention can also be used when the values of 
sprite_enable and video_object_layer_shape are different. 
People skilled in the art will appreciate that as long as the 
(no_of_sprite_warping_point == 0 or 1), this invention ap- 
plies even when the values of sprite.enable and 
video_object_layer_shape are different from the above and 
the equations become different. 

[0053] pig. 9 shows a flowchart describing a method of process- 



ing an incoming coded video bit stream comprising a plu- 
rality of frames according to the present invention. Each 
frame may include a plurality of macroblocks encoded us- 
ing block-matching motion compensation and/or a plu- 
rality of macroblocks encoded using global motion com- 
pensation. The flowchart contains the following steps: 
[0054] step 900:For each frame received in the incoming video 
stream, convert the global motion parameters associated 
with the frame into a luminance global motion vector and 
a chrominance global motion vector. Store the luminance 
global motion vector and the chrominance global motion 
vector for later global motion compensation luminance/ 
chrominance interpolation operations. Proceed to step 
902. 

[0055] step 902:When decoding a current macroblock, determine 
whether the current macroblock is encoded using block- 
matching motion compensation or global motion com- 
pensation. If block-matching motion compensation, pro- 
ceed to step 904, otherwise if global motion compensa- 
tion, proceed to step 908. 

[0056] step 904:Extract the macroblock motion vectors stored in 
the current macroblock. 

[0057] step 906:Perform the luminance and chrominance bilinear 



interpolation operations according to the macroblock mo- 
tion vectors extracted in step 904. Use a half-pel resolu- 
tion for both luminance and chrominance. Processing is 
complete. 

[0058] step 908:Perform the luminance bilinear interpolation op- 
eration according to the luminance global motion vector 
stored in step 900. Use half-pel resolution and proceed to 
step 910 when finished. 

[0059] step 910:Perform the chrominance bilinear interpolation 
operation according to the chrominance global motion 
vector stored in step 900. Use quarter-pel resolution and 
when finished, processing is complete. 

[0060] | n contrast to the prior art, the present invention performs 
bilinear interpolation operations on macroblocks encoded 
using global motion compensation according to a lumi- 
nance global motion vector and a chrominance global 
motion vector so that the block-matching motion com- 
pensation and global motion compensation can be inte- 
grated into a single unit. The luminance/chrominance 
global motion vectors are converted from a set of global 
motion parameters transmitted with each frame in the in- 
coming coded video bit stream. For an MPEG-4 compliant 
coded video bit stream having GMC macroblocks with the 



no_of_sprite_warping_point parameter set to either 0 or 1, 
the global motion compensation calculations can be sim- 
plified to resemble the interpolations operations normally 
performed for block-matching. The difference is that for 
the chrominance interpolation operations for macroblocks 
encoded using global motion compensation, a quarter-pel 
resolution is used. For luminance and chrominance inter- 
polation operations for macroblocks encoded using 
block-matching motion compensation and for luminance 
interpolation operations for macroblocks encoded using 
global motion compensation, a half-pel resolution is 
used. 

[0061] it should be noted that when a frame includes only mac- 
roblocks encoded using block-matching motion compen- 
sation, there may be no global motion parameters associ- 
ated with the frame. In such case, the present invention 
will not perform the conversion process of converting the 
global motion parameters into a luminance global motion 
vector and a chrominance global motion vector because 
there are no global motion parameters associated with in 
the frame. 

[0062] Those skilled in the art will readily observe that numerous 
modifications and alterations of the device may be made 



while retaining the teachings of the invention. Accord- 
ingly, the above disclosure should be construed as limited 
only by the metes and bounds of the appended claims. 



